Multispectral Texture Mapping for Telepresence and Autonomous Mobile Robotics

The aim of this paper is to describe the data-fusion acquired from a CCD camera and a thermal imager. The fusion is realized by means of spatial data from a TOF camera to ensure ”natural” representation of a robot’s environment; thus, the thermal and CCD camera data are comprised in one stereo image presented to a binocular, head-mounted display. The data acquisition is performed using a sensor head, which is placed on an Orpheus-X3 robot; both the head and the robot were developed by our working group. The head contains five matrix sensors: a pair of CCD cameras, a pair of thermal imagers, and one TOF camera. After the geometrical calibration of each sensor, the positions of the sensors in 6DOFs are computed. The corresponding data from the CCD camera and the thermal imager are subsequently determined via homogeneous and perspective transformations. The result consists in an image containing aligned data from the CCD camera and the thermal imager for each eye. TOF camera calibration and its impact to the precision of fusion is described. Thermal imager color misbalance caused by mutual temperature shift is described and solution is shown.